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We endorse the comment on our recent paper [EnBlin and Weig, Phys. Rev. E 82, 05 1 1 12 (2010)] by Iatsenko, 
Stefanovska and McClintock [Phys. Rev. E 85 033101 (2012)] and we try to clarify the origin of the apparent 
controversy on two issues. The aim of the minimal Gibbs free energy approach to provide a signal estimate is 
not affected by their Comment. However, if one wants to extend the method to also infer the a posteriori signal 
uncertainty any tempering of the posterior has to be undone at the end of the calculations, as they correctly point 
out. Furthermore, a distinction is made here between maximum entropy, the maximum entropy principle, and the 
so-called maximum entropy method in imaging, hopefully clarifying further the second issue of their Comment 
paper. 


Iatsenko, Stefanovska and McClintock [1] point out two 
possible sources of confusion about how the minimal Gibbs 
free energy method which was introduced in our paper [2], 
has to be used. 

The first point of confusion concerns the usage of temper- 
ature in the Gaussian approximation of a high dimensional 
probability density function. This was introduced to enable us 
to emphasize regions near (temperature T < 1 ) or more distant 
from (T > 1) the maximum by giving them a larger weight in 
the approximation. Our original claim is still correct in that 
the signal mean can be read off approximately from the mean 
of this tempered Gaussian. If the uncertainty information is 
also extracted from the tempered Gaussian, the tempering has 
to be undone at the end of the calculation, as was pointed out 
correctly in [1]. This was actually implicitly done in our pa- 
per in Eq. (44) and very indirectly indicated in the sentence 
enclosing this equation. However, we fully agree with [ 1] that 
this important fact should have been spelled out more promi- 
nently. 

The second point is the slightly deceptive usage of the term 
maximum entropy (ME) in our paper, which is defined there as 
the unconstrained maximum of the Boltzmann/Shannon en- 
tropy. This term should not be confused with the maximum 
entropy principle (MEP) [3], which is the constrained maxi- 
mization of such an entropy, a distinction correctly made by 
[1]. Our aim in using the terminology of ME was to highlight 
the importance of incorporating phase-space volume factors 
correctly into inference schemes, which counteracts the at- 
traction of the posterior maximum and thereby impedes over- 
fitting. 

We might have given the impression that we were arguing 


against the usefulness of the MEP by discussing critically, but 
only briefly, the often used maximum entropy method (MEM) 
in imaging. In hindsight, it might have been wiser to put this 
discussion into a separate publication, so as not to risk further 
confusion. In any case, we want to explain the difference here. 
The MEM imaging can be derived from the MEP in the spe- 
cific case in which the image is composed of a nearly infinite 
number of uncorrelated and in principle distinguishable flux 
elements of an infinitesimally small strength each |4]. In this 
case, each possible image can be attributed to an internal im- 
age entropy, due to the number of ways the flux elements can 
be redistributed among pixels without changing the image. 
MEM can be shown to be a maximum a posteriori method, 
with the internal entropy prior suppressing large flux values 
stronger than exponentially [2]. Although the conditions un- 
der which MEM is optimal are rather artificial and rarely met 
for the problem at hand, its application is widespread. How- 
ever, ME and the MEP are more general, in that they not only 
deal with the internal entropy of a single image realization, 
if this can be defined, but with the full entropy of the phase 
space of all possible images. The discussion of this issue in 
our paper might have given the impression that we wanted to 
argue against the MEP, while we merely criticized the unjusti- 
fied usage of MEM image restoration. We want to emphasize 
here, that we consider the MEP to be a correct and important 

concept. 
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